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ABSTRACT 

This paper reports on the effects of structuring 
problem solving activities of novices in the domain of physics in a 
way consistent with the problem solving approaches used by experts. A 
total of 42 undergraduate students who completed a first semester 
physics course and received a grade of B or better participated in 
this study. The treatment involved five 1-hour sessions during which 
subjects solved a total of 25 classical mechanics problems using a 
hierarchical, computer-based, problem-analysis environment called the 
Hierarchical Analysis Tool (HAT) . Two types of tasks were 
administered before and after treatment: (1) a problem-categorization 
task; and (2) a problem-solving task. The results indicate that the 
hierarchical approach to problem solving as exemplified by the HAT 
helped students to shift their decision-making criteria for problem 
categorization from one based on surface features toward one based on 
deep structure. Two implications from this study were discussed. 
Tv/elve references are listed. (YP) 
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INTRODUCTION 

Becoming an expert in a given domain takes substantial time and effort. 
This raises an interesting question: For a given individual possessing 
aptitude in some domain, is becoming an expert simply a function of time and 
effort, or can the path toward expertise be made more efficient? The research 
reported in this article reports, on the effects of structuring the problem 
solving, activities of novices in the domain of physics in a way consistent 
with the problem solving approaches used by experts. The focus of our 
investigation was to assess the possibility of promoting expert-like behavior 
among novices by constraining them to follow an expert-like approach to 
problem analysis. 

That novices and experts store and use domain-specific knowledge in 
distinctly different ways is the consensus of a number of studies in such 
diverse fields as chess (Chase & Simon, 1973), computer programming (Ehrlich & 
Soloway, 1982), electrical circuits (Egan & Schwartz, 1979), and classical 
mechanics (Larkin, 1979). Experts tend to store information in hierarchically 
structured clusters related by underlying principles or concepts. When 
attempting to solve a problem, experts initially focus on the principles and 
heuristics that could be applied to solve that problem (referred to as deep 
structure cuing). In contrast, the knowledge base of novices ia less 
structured and has fewer interconnections. When solving problems, novices do 
not focus on principles or heuristics that could be used to construct a 
solution strategy; rather, they focus on objects and descriptor terms in the 
problem (called surface features) and then look for the actual equations that 
could be manipulated to yield an answer (Chi, Feltovich & Glaser, 1981; 
Larkin, McDermott, Simon & Simon, 1980; Mestre Gerace, 1986). 

Studies in the domain of physics (Eylo.n & Reif, 1984; Heller & Reif, 
1984) suggest that instructional approaches that impose a hierarchical, 
expert-like organization both on information, and on problem solving 
heuristics result in improved problem solving and recall performance among 
novices. Despite these, and other related findings curr^ont instructional 
practice does not emphasize hierarchical approaches to knowledge organization 
or to problem solving. Consequently, much of the the expert's tacit knowlodge 
remains a secret to the novice until she or he discovers it on her own. 

In the present study novices actively participated in problem solving 
activities which were structured to reflect our best understanding of how 
physics experts analyze problems. The treatment involved five one-hour 
sessions during which subjects solved a total of 25 classical mechanics 
problems using a hierarchical, computer-based, problem-analysis environment 
called the Hierarchical Analysis Tool. The effectiveness of this treatment 
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FIGURE 1. Hierarchical Analyzer Menui i& ChoIcM for Problem 1 
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PARTICIPANTS IN EXPERIMENTS 1 AND 2 

Subjecta 

had ro!n!:«r5"°K""^!''^r'*"^''^ students at the University of Massachusetts who 
had completed the first semester physics course for majors or for engineers, 
and received a grade of B or better, participated in this study. T^e subjects 
fmi^dSuars!" experimental sessions, for which, they were paid 

Groups 

On the basis of pretest scores, the 42 subjects were divided into three 
treatment groups of U subjects each. Each of the three groups received the 
sajue 25 problems over the course of the treatment. Subjecta solved five 
problems in each of five sessions over approximately three weeksj the 
treatment problems were representative of problems the aubjocta had 
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sncouat^red in their course and covered the major topics in a beginning 
clasaical mechanics course. The •'HAT-group'* used the Hierarchical Analysis 
Tool in solving the 25 treatment problems. The ••EST-group'' used the Equation 
Sorting Tool in solving the treatment problems, while the "T-group'* used a 
homework-style approach in solving the treatment problems and were free to 
refer to the textbook that they had used in their course to solve the 
problems. 



EXPERIMENT 1; SIMILARITY JUDGMENT TASK 
We designed a similarity Judgment task (Hardiman, Dufresne & Mestre, 
1987) in which subjects were to decide which of two comparison problems would 
be solved most similarly to a third model problem. Surface feature and deep 
structure similarity to the model problem were varied systematically, allowing 
us to investigate whether subjects were more likely to focus on deep structure 
similarity as a basis for categorization following treatment. Since the 
initial decision that must be made in the HAT concerns the principle to be 
applied, we hypothesized that the HAT-group would be more likely to focus on 
deep structure after treatment than either of the two control groups. 

This task contained 20 items.. Each task item was composed of three 
elementary mechanics problems, each of which was three to five lines long and 
contained only text (no pictures or diagrams). For each item, one of the 
three problems was identified as the model problem , while the other two were 

compariso n problems . The subjects were to indicate which of the two 
comparison problems they believed ""would be solved most similarly to the 
model problem. 

A comparison problem could share different attributes with its nodel 
problem. Four types of comparison problems wore designed that matched the 
model problem in: 1) surface features, meaning that the objects and descriptor 
terms that occur in both problems are similar, 2) deep structure, meaning that 
the physical principle that could be applied to solve both problems is the 
same, 3) both surface features and deep structure, or 4) neither surface 
features nor deep structure. These four types of comparison problems were 
termed S, D, SD, and N, respectively. 

The comparison problems were paired such that only one of the two 
comparison problems matched the model problem in deep structure. This 
constraint led to four types of comparison problem pairs: 1) S-D, 2) S-SD, 
3) N-D, and 4) N-SD. Assuming a categorization scheme based strictly on 
surface features, the following pattern of performance was predicted: 1) S-D- 
OX deep structure choices, 2) S-SD: SOX deep structure choices (both choices 
are equally good in terms of matching the model problem on surface features), 
3) N-D: SOX deep structure choices (either alternative is equally «bad** in 
terms of matching the model problem on surface features), and 4) N-SD: 100% 
deep structure choices (a surface feature match to the model problem will also 
mean a deep structure match). In contrast, assuming a deep structure 
categorization scheme would result in iCOX deep structure choices in all four 
pairings. ^ 

The task was presented via computer. The subject was told to read 
carefully the model problem and two comparison problems, and to respond by 
pressing one of two keys. The items were presented in random order, with no 
limit imposed on time to respond. After every 5 items, the subject was given 
the opportunity to take a brief rest. Most subjects completed the task within 
4S minutes. The same task was presented after the subjects had completed the 
five treatment sessions. 
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Re3ult3 and Dlgffii<i .<5<nn 

r oV"^ performances of the 42 subjects were compared in a 3 (Treatment 
T^iXl) i 5lSoi:rpi'S." ' (Times: pre , ,olt) x 4 (Comp^rCoTrro'blam 
«h^tw%\ u*i V Problems) analysis of variance. The focal question was 
sJri^tur. ^ tre£tment would promote a shift toward reliance on de.p 
llnrl r, °" ^""^^"^ features. The results indicate an 

nf di??«irn "^r^" '° l^^^ question. In the pre-judgment task, there were 
■iere fLn!?L „?%T:r '''' " However, there 

Wol™^„f r '^^^^^-^^^ the three group- in the amount of 

F?2"J) - l28Tp'"2.'° °" the judgment task (see Table 1), 

th«<r I*"^ «AT-group was the only group to show anjr indications of improvement- 
their improvement waa atatiatlcally «lgniflcant7(F(l. 13^-5. 20. p-'J^r in ' 
contrast, the mean performance of the EST-group remalied'the Le. whUe tSe 

profotTr:hJft'?' ""TT that thetli does 

trlTtllnt J toward the use of deep structure, while the two control 
i^t^ Zro T' .""^^^ consistent across Comparison Problem Types 

L™^. T K?" °^ ^ percentage points in each of the four 

DaTr?nea°? '[jJ " P^^'^^'^^^' ^^^^^ Improvement was significant for the S-D 
?our J!!tit P"-t°-Po«t improvement). t(13) - 3.12, p..0324 (adjusted for 
featu^ef^^d J t" given that the S-D items, where sj^face 

features and deep structure are in direct competition, present the most 
difficulty for novices (Hardiman, et al., 1987). 

Table I'. Pre- and Post-Judgment Task Percent Correct for the 3 GrouDs 



Group 
HAT 

61% 61% 
T 62% 



pre-treatment post-treatment 
56% 



Total 



62% 



EXPERIMENT 2: PROBLEM SOLVING TASK 
t..t. the effect of treatment on problem solving, two equivalent 

lev Till '""^ ^'y'^ °' « traditional ilnal';xrfor fresLn 

thr es on ?he Tr " °' '""'^ ^"''^"^^ ""i-'' one form o^ 

Jh: f": ::t^^^;"rt^";r:sr:e;^r:e^d"LVt^":^ 

Iriitn t, c solution and the remaining 3 problems requirinR the 

sTeTAl IIXTJ:,'^'''''-' ^"'^"'^ ""^ ''^'^ approximateJ^oi^^^S: to 

to the"post-?est' 'f' subjects would improve in performance from the pre- 

dur?na I ^' subjects Would have practiced solving problems 

aSp ;?n« th«" J S'"!' HAT-group was capable of adopting and 

exE Lt'b ? er pre'- o T^''^' °" Post-test, we might expeJ them to 

exniDic Detter pre-to-post improvements than the EST- and T-groups. 

Results and Discussion 

score S'ai^t'eVSIf'ffLl'Li"''"'"?'""^ physicists. Whenever the 

score on an item differed between the graders, the subject's solution was 



reevaluated and a score was determined by consensus. The pre- and post-test 
scores are shown in Table 2. All three groups increased about 10 percentage 
points, mainly due to improvements on the single-principle profalftras. Although 
pre-to-post improvements were statistically significant (F(l,39)-21.25, 
p<.0001), no one group improved significantly more than any other group. This 
suggests that, at least for treatments lasting a short period of time, the 
improvement on problem solving was primarily due to practice in problem 
solving in general, not to any specific treatment. 

Table 2: Percent Correct (S.D.) in Pre-, and Post-Problem-Test 



Group Pre-Test 



Post-Test 



HAT 29.4 (20.1) 41.3 (17.5) 

EST 36.4 (25.8) 44.9 (25.9) 

5 31.6 (24.6) 44.4 (24.4) 



Given that the improvement of the HAT-group was not significantly better 
than that of the two control groups, we might ask whether the HAT-group was 
able to use the HAT appropriately? Our data indicate the answer is no: an 
analysis of the HAT-group 's key-stroke data indicates that subjects were able 
to carry out appropriate analyses using the HAT on less than half of the 
treatment problems. Thus, the full potential of the HAT approach for 
improving problem solving skills cennot be fully evaluated until we ensure 
that subjects adopt the approach incorporated in the HAT. 

GENERAL DISCUSSION 
The results of Experiment 1 indicate that the hierarchical approach to 
problem solving, as exemplified by the HAT, helps students to shift their 
decision making criteria for problem categorization from one based on surface 
features toward one based on deep structure. We speculate that use of the HAT 
promotes this shift because it highlights the importance of applying 
principles to solve problems by asking subjects to select the applicable 
principle in the first menu they encounter. Even if the user is not able to 
answer all of the subsequent questions in the analysis correctly, the 
principle to be applied to obtain a solution may still be recognized as 
primary. 

However, the current implementation of the hierarchical approach did not 
lead to significantly more improvement in problem solving than a "homework 
style approach. Assuming the hierarchical approach La a potentially powerful 
tool for improving problem solving, there are two main reasons why the HAT 
treatment failed to yield more significant improvements in problem solving. 
First, subjects were not able to internalize the approach implicit in the HAT, 
since thay had no way to gauge whether or not th&y war. using the HAT 
appropriately. It appears that feedback and coaching are necessary 
ingredients to help novices assimilate the HAT's expert-like approach 
(Collins, Seely Brown & Newman, in press). Second, the treatment was 
relatively short., and therefore it is unrealistic to expect dramatic 
reorganization of declarative and procedural knowledge after using the HAT for 
only 5 hours. 

Two implications appear to stand out. First, hierarchically structuring 
the problem analysis activities of novices holds promise for promoting expert- 



ERIC 



8 



like behavior among novices. However, simply partaking in expert-like problem 
solving activities is not sufficient to promote dramatic improvements in a 
complex task such as problem solving— we need to guatantee that novices 
actually adopt the hierarchical approach before we can evaluate its full 
potential. Second, problem solving assessments may not be the most sensitive 
for measuring modest shifts toward expertise. Other measures, such as problem 
categorization, appear to be more sensitive assessnents of subtle shifts 
toward expertise. 



Cognitive PsycholoRV , 



REFERENCES 

Chase, W.G. & Simon, H.A. (1973). Perception in chess, 
4, 55-81. 

Chi, M.T.H.,~Feltovich, P.J. & Giaser, R. (1981). Categorization and 
representation of physics problems by experts and novices. 
Cognitive Science . 5, 121-152. 
Collins, A., Brown, J.S. & Newrain, S.E. (in press). Cognitive apprenticeship: 
Teaching the craft of reading, writing and mathematics. In L. 
Resnick (Ed.), Cognition and Instruction; Issues and Agendas . 
Hillsdale, NJ: Lawrence Erlbaura Assoc. 
Dufresne, R. , Gerace, W. , Hardiraan, p. & Mestre, J. (1987). Hierarchically 
structured problem solving in elementary mechanics: Guiding 
novices' problem analysis. Proceedings of the Second 
Internationa l Seminar on Misconceptions and Educational Strategies 
in Science and Mat hematics (Vol. Ill, pp. 116-130). Cornell 
University, Department of Education: Ithaca, NY. 
& Schwartz, B.J. (1979). Chunking in recall of symbolic drawings. 
Memory & Cognition . 2. 149-158. 

& Soloway, E. (1982)7 An empirical investigation of the tacit plan 
knowledge in programming. Research Report tf236. Department of 
Computer Science, Yale University. 

& Relf, F. (1984). Effect of knowledge organization on task 
performance. Cognition & Instruction . 1, 5-44. 
.T., Dufresne, R. & Mestre, J. (1987). Physics novices' judgments 
of solution similarity: When are they based on principles? 
Proceedings of the Second International Seminar on Misconceptions 
and Educational Stra tegies in Science and Mathematics (Vol IlTi; 
pp. 194-202). Cornell University, Department of Education: Ithaca, 
NY. 

& Reif, F. (1984). Prescribing effective huraan problem solving 
processes: Problem description is physics. Cognit ion & 

Instruction ^ 1^, 177-216. — 

H. (1979). Information processing models in science instruction. 
In J. Lochhead & J. Clement (Eds.), Cognitive Process Instruction . 
Hillsdale, N J : Lawrence Erlbaura AssocT^ ! 
Larkin, J.H., McDermott, J., Simon, D. & Simon, H. (1980). Models of 

competence in solving physics problems. Cognitive Science, 4. 
317-345. — 

Mestre, j.P. & Gerace, W.J. (1986). Studying the problem solving behavior of 

experts and novices in physics via computer-based problem-analysis 
environments. Program of the Eighth Annual Conference of the 
Cognitive Science Society (pp. 741-746). Hillsdale, NJ; Lawrence 
Erlbaura Assoc. 



Eg an, D. 
Ehrlich, K. 

Eylon, B.S. 
Hardiraan, 



Heller, J.I, 



Larkin, J* 



